Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Deep neural network model acceleration method based on tensor virtual machine
Yunfei SHEN, Fei SHEN, Fang LI, Jun ZHANG
Journal of Computer Applications    2023, 43 (9): 2836-2844.   DOI: 10.11772/j.issn.1001-9081.2022081259
Abstract262)   HTML10)    PDF (3331KB)(126)       Save

With the development of Artificial Intelligence (AI) technology, the Deep Neural Network (DNN) models have been applied to various mobile and edge devices widely. However, the model deployment becomes challenging and the popularization and application of the models are limited due to the facts that the computing power of edge devices is low, the memory capacity of edge devices is small, and the realization of model acceleration requires in-depth knowledge of edge device hardware. Therefore, a DNN acceleration and deployment method based on Tensor Virtual Machine (TVM) was presented to accelerate the Convolutional Neural Network (CNN) model on Field-Programmable Gate Array FPGA), and the feasibility of this method was verified in the application scenarios of distracted driving classification. Specifically, in the proposed method, the computational graph optimization method was utilized to reduce the memory access and computational overhead of the model, the model quantization method was used to reduce the model size, and the computational graph packing method was adopted to offload the convolution calculation to the FPGA in order to speed up the model inference. Compared with MPU (MicroProcessor Unit), the proposed method can reduce the inference time of ResNet50 and ResNet18 on MPU+FPGA by 88.63% and 77.53% respectively. On AUC (American University in Cairo) dataset, compared to MPU, the top1 inference accuracies of the two models on MPU+FPGA are only reduced by 0.26 and 0.16 percentage points respectively. It can be seen that the proposed method can reduce the deployment difficulty of different models on FPGA.

Table and Figures | Reference | Related Articles | Metrics